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In this paper, delay-free, low complexity, joint source-channel coding (JSCC) for transmission 



of two correlated Gaussian memoryless sources over a Gaussian Multiple Access Channel (GMAC) is 
considered. The main contributions of the paper are two distributed JSCC schemes: one discrete scheme 
based on nested scalar quantization, and one hybrid discrete-analog scheme based on a scalar quantizer 
and a linear continuous mapping. The proposed schemes show promising performance which improve 
with increasing correlation and are robust against variations in noise level. Both schemes exhibit a 



, constant gap to the performance upper bound when the channel signal-to-noise ratio gets large. 

t-^- I. Introduction 

in: 

In a point-to-point communication system, for a memoryless Gaussian source-channel pair 
of equal bandwidth, it is well known that a simple encoder that scales its incoming signal to 
satisfy the channel power constraint and minimum mean square error (MMSE) decoding at the 
^ ■ receiver achieves the information theoretical bound optimal performance theoretically attainable 

I (OPTA) [IJ. This linear approach, which is often referred to as uncoded transmission in the 

literature, constitutes a very simple joint source-channel coding (JSCC) scheme due to its low 
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complexity and zero coding delay. Separate source and channel coding (SSCC), as summarized 
by the separation theorem also achieves OPTA, but requires infinite complexity and delay. 

In this paper we investigate a multipoint-to-point problem, where two memoryless and inter- 
correlated Gaussian sources are transmitted over a memoryless Gaussian multiple access channel 
(GMAC). There are mainly two cases to consider for such a network: 1) Recovery of the common 
information shared by the two sources. 2) Recovery of each individual source. 

Case 1) was studied in e.g., [|3l, flU, [|5l. Distortion lower bounds was derived and for the case 
of equal (source) variance, it was shown that these bounds are achieved by uncoded transmission 
when the transmit power of all encoders are equal. It was also shown that any distributed SSCC 
scheme is sub-optimal, except when the sources are uncorrelated. 

Case 2) was recently studied in [[6l . The distortion lower bound was derived by allowing full 
collaboration between the encoders and thus converting the multi-point-to-point into a point- 
to-point communication problem. The best possible performance can then be determined by 
bounding the rate-distortion region of the bivariate Gaussian [|6l Theorem III.l] by the GMAC's 
sum rate Theorem IV. 1]. Closed form solutions were given for the symmetric case [[6l 
Corollary IV. 1], i.e. when the average distortion in the reconstruction and the transmit power 
are equal for both sources. It was shown that uncoded transmission achieves the distortion 
lower bound up to a certain channel signal-to-noise ratio (SNR), depending on the correlation 
between the two sources. In order to get close to the distortion lower bound in general, the 
authors further proposed a nonlinear hybrid scheme that superimposes a rate optimal (infinite 
dimensional) vector quantizer (VQ) and uncoded transmission. The hybrid scheme was shown 
to be optimal at high and low channel SNR, while a small gap remains for other SNRs. The 
authors also examined distributed SSCC. Just as in case 1), SSCC is sub-optimal except when 
the sources are uncorrelated. Contrary to case 1), optimality of uncoded transmission for case 
2) is restricted and infinite complexity and delay JSCC is required to close in on the bounds in 
general. 

This result prompts an important question: What happens if we impose a strict complexity 
and delay constraint for case 2)? The main objective of this paper is to provide answers to this 
question. In particular, we want to find well performing, simple and implementable schemes 
with zero coding delay, just as that offered by uncoded transmission. That is, simple distributed 
nonlinear mappings that offer better performance than uncoded transmission outside the SNR 
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domain where uncoded transmission is optimal. 

It is important to keep in mind that when assessing performance of a JSCC scheme designed 
under a zero-delay constraint, one must expect a significant backoff from the distortion lower 
bound derived in [6], as it is based on infinite block length. We therefore briefly address known 
zero-delay cooperative^ encoding schemes to provide indications on where the bound may lie 
for any zero-delay distributed scheme. 

The rest of the paper is organized as follows: In Section UIl the problem formulation and 
relevant bounds are given. Zero-delay mappings for cooperative encoding are also introduced. In 
Section Unl we present two distributed schemes: A discrete (digital) mapping based on Nested 
Quantization [[7J, and a hybrid discrete-analog scheme using a scalar quantizer and a limiter 
followed by a linear coder. Both schemes are optimized. Extensions to more general cases are 
discussed and we show that the suggested schemes exhibits a constant gap to the bound when 
SNR— 7- oo. In Section HyI the proposed schemes are simulated and compared to the performance 
bounds and other relevant schemes under both average- and equal transmit power constraints. 
We summarize the results in the paper in Section |V] and give some future research directions. 



II. Problem statement and upper bounds 
The communication system under consideration is shown in Fig. [TJ 
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Fig. 1. Two correlated Gaussian memoryless sources transmitted on a GMAC with both sources reconstructed at the receiver. 



'By cooperation we mean that both source symbols are available at both encoders without any additional use of resources. 
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A. Problem statement 

The sources xi and X2 are assumed to be zero mean Gaussian random variables Xm = v+u)m ~ 
M{0, crl^), m = 1,2, where v ~ A/'(0, cr^^) is the common information for both sources and 
Wm ~ A/'(0, c"^,^) is unique to each source, where v and Wm are independent. Further we assume 
that cr^^ = cr^^, implying that the variances cr^^ = = '^x- The 2x2 covariance matrix Cx 
will then have on the diagonal and a^px on the off-diagonal elements with eigenvalues 
Ai = (T^(l + px) and A2 = cr^(l — p^;), where px = E[xiX2]/cr^ = cr'^Jcrl denote the correlation 
between xi and X2. The joint probability density function (pdf) is given by 

Px(x) = Px(Xl, X2) = \ ^ g-|x^C-^x (J) 

with equal marginals ^^(a^m)- 

We denote the zero-delay encoding functions by fm{xm),m = 1,2. The average transmit power 
from encoder m is then = E[|/m(xm)p], m = 1, 2. The encoder outputs are transmitted on 
a memoryless MAC with additive Gaussian noise n ~ A/'(0, cr^) with pdf Pn{n). 

We assume ideal Nyquist sampling and an ideal Nyquist channel where the sampling rate of 
each source is the same as the signalling rate of the channel. We also assume ideal synchroniza- 
tion and timing between all nodes. 

The received signal 

z = fi{x^) + f2{x2)+n (2) 

is passed through the decoding functions gi, g2 to produce an estimate of each individual source 
xi,X2. We use the mean-squared-error distortion criterion, and define the average end-to-end 
distortion as: 

D = ^{Di + D2) = ^(E{|xi-xi|2} + E{|x2-X2p}). (3) 

Our goal is to design the mapping functions {fm.,gm),rn = {1,2}, under a given power constraint, 
so that D is minimized. 

B. Performance upper bound 

For the above defined communication system, we consider both an average and equal transmit 
power constraint. The reason for the former is that the distributed schemes we propose in 
Section Un] have asymmetric encoders resulting in Pi > ^2- It is therefore convenient to optimize 
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our schemes under an average transmit power constraint P, where Pi + P2 = 2P. If an equal 
transmit power constraint Pi = P2 = P is imposed a loss is expected (shown in Section IIV-BI) . 

The performance upper bound for the symmetric case Di = D2 = D and Pi = P2 = P, 
expressed in terms of the signal-to-distortion ratio (SDR) is 



where D is the distortion lower bound from ^ Corollary IV. 1]. Although this bound was derived 
for Pi = P2 = P it is also a bound for an average transmit power constraint by simply substituting 
P = (Pi + -P2)/2. The term "channel SNR" refers to P/cr^ in the rest of the paper. Uncoded 
transmission achieves the bound given by the upper equation in [!6l Corollary IV.3], and 
corresponds to the SNR where only the common information v can be recovered at the decoder. 

C. Delay-free JSCC for cooperative encoders 

Since collaboration makes it possible to construct a larger set of encoding operations, including 
all distributed strategies, the performance of distributed coding schemes are upper-bounded 
by those that allow collaboration when properly optimized. The performance of the proposed 
distributed JSCC scheme relative to the performance upper-bound described in [6] is one such 
example. 

In the case of zero delay, the corresponding optimal collaborative encoding operation is the 
optimal mapping — j- M from source to channel space, which minimizes at a given power 
constraint. Finding the optimal structure of such a mapping is a problem yet to be solved. We 
can, however, get an idea of how collaborative encoders may perform from known schemes that 
perform close to the bounds. Examples on schemes with excellent performance are Shannon- 
Kotel'nikov mappings (S-K mappings) (HI, flU and Power Constrained Channel Optimized Vector 
Quantizers (PCCOVQ) [[TOl. When the number of centroids in the PCCOVQ is large they are 
very similar to S-K mappings, we therefore refer to both of these as S-K mappings in the 
following. S-K mappings have previously been optimized for memoryless Gaussian sources and 
channels when M source symbols are transmitted on N channel uses [[HI, [[T0l -[[T2 | . 

For the problem at hand, if we treat the collaborative encoders as one, and the two sources 
as two components of a Gaussian vector source, we can apply S-K mappings with M = 2 and 
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= 1 directly. This collaborative zero-delay scheme can not be applied to the distributed case 
as their operation relies on knowing both source symbols simultaneously at each encoder [8J, 
IfTOl . Nevertheless, we can use the collaborative schemes as benchmarks and see how much loss 
one may expect with distributed encoding. This will be illustrated in Section |IVl 

in. Distributed zero-delay schemes 

Since we want to split the two interfering sources at the receiver, we have to construct our 
JSCC schemes accordingly. A discrete (digital) approach that achieves this purpose is Nested 
Quantization (NQ) introduced in |l7J with sequential decoding at the receiver. 



A. Nested Quantization 

Our NQ scheme consists of two uniform scalar quantizers, one for each encoder. Without loss 
of generality we choose encoder 2 to be the nested quantizer. Fig. [21 shows the NQ encoding 
process. The encoding functions fi{xi) and f2{x2) arc 



yi = fi{xi) = a - c- ii{xi), y2 = f2{x2) = a ■ i2{x2). 
im{xm) denotes the quantization process that returns an index in Z: 



^m. \ Xrt 



Xr, 



m = 1, 2, 



(5) 



(6) 



where [■] denotes rounding to the nearest integer and A denote the quantization step. Further, 
for encoder 2, the nested quantizer is invoked by 

i2ix2 



i2{x2) = i2ix2) - C 

as depicted in Fig. 12 a). I.e., 22 is the mapping 

c- 1 



c e N, 



^2 



(7) 



(8) 



2 ' 2 ' 2 

That is the whole real line is mapped into a fixed bounded interval of discrete values. In order 
to identify each encoder output after they have been summed over the GMAC, ii must be scaled 
by c. This is illustrated in Fig. [2b) after the sum. Now both sources can be reconstructed at 



The following equations describe midthread quantizers with odd c. Similar equations can be derived for midrise quantizers. 



April 5, 2012 



DRAFT 



6 




Fig. 2. NQ encoding process, (a) The nested quantizer 12 for c = 7 and A = 1. The whole real line, representing X2, is 
mapped to a fixed number of centroids (here 7 values) inside a finite interval, (b) How encoding is performed when c — 3. The 
gray arrows indicate that the structure continues. 



the receiver through sequential decoding. The parameter a is set to satisfy the power constraint. 
The average transmit power from each encoder is 

00 {c-l)/2 

Pi = cV ^2 = a' Yl P<^Sl (9) 

il=-oo i2=-(c-l)/2 
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where Pr(-) denote the probability for the event inside the parentheses. 

At the decoder we first recover the indices from encoder 1, ji{z), using the maximum 
likelihood (ML) estimate (based on the decision values seen in Fig. [21) 



arg min ||/i(jA) 



(10) 



j2{z) from encoder 2 is then detected after subtracting fi{ji{z)A) from the channel output 

j2{z) = arg min \\z - fi{ji{z)A) - /2(jA)||2. 



j&--\j\<^ 



(11) 



As seen from Fig. [2ta), j2iz) corresponds to an infinite number of estimates of 22- It is therefore 
not possible to invert the mapping ([8]) from j2 to j2 based on j2{z) alone. But the decoder also 
has access to ji{z), which due to the correlation contains information about X2, and hence also 
j2{z). To determine the most likely interval (see Fig. lUa)) that j2 belongs to, we consider the 
maximization of ([U). Given xi = ji(z)A and that argmaXxPx(x) = argminx (x^C~^x) the 
most likely value of j2 can be approximated by 



arg mm 

J2 







T 






( 






Ji(^)A 


) 




j2A 




J2A 





(12) 



where j2 G {j2iz) + kc,k e Z}. Solving (fT2l) with respect to k we get 

Pxjl{z) -j2{z) 



k = max 



.0 



(13) 



By choosing the constant c appropriately with respect to p^, the decoder will with high probability 
be able to invert the mapping Z2 ^2 correctly. The larger p,j. is, the more accurately ji (z) will 
help in telling which interval ^2 is in, implying that c can be made smaller the larger is. 
In order to minimize the MSE, we further calculate 



Xm = E{Xm\jl{z)j2iz)}, 



m 



1,2. 



(14) 



To design the optimal NQ, we need to determine the A, c and a that minimize D under a 
given power constraint. Since the two encoders are asymmetric. Pi > P2, as seen from We 
must solve the following optimization problem 



min D, 

A,c,a:Pi+P2<2P 



(15) 



where D is the average end-to-end distortion and P is the average transmit power. 
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Distortion calculation: In order to simplify notation when deriving the distortion, we intro- 
duce the following auxiliary variables 

Xm = E{xm\ii, 12}, Xm = E{f „|zi, ^2}, m = 1, 2. (16) 

Xff^i is the quantized source, Xj-n th^ quantized source after the map Z2 and is as defined 
in (fT4l) . The Appendix shows that the per source distortion Dm can be split into three terms: 

(17) 

ig^rn is the quantization distortion given by 

£q,m = J J P^{xu X2){xm " Xz(zi(xi), Z2 (a:2) ) )^dxidx2 ^ (18) 

where the last approximations is valid for small A. Ec,m represents the distortion from the 
inversion of im im, resulting when the wrong interval in Fig. Oa) is detected at the decoder 



PAxi,X2){Xm{iliXi),i2{x2)) - (zi (Xi ) , ^2 (^2) ) ) dXidX2 

^ ~ . (19) 

2_^Pr{li,l2){Xm{il,i2) - Xm{%i,%2{l2))y 
h,i2 



\2 



where i2{i2) represents the mapping from Z2 to 12 in (|7]). the distortion due to channel 

noise, is given by the following equation 

^nm = PAxi,X2)p{z\ii{Xi),i2{x2)){Xni{il{Xi),i2{x2)) - Xm{jl{z) , j2{z)))'^dzdXidX2 



= ^^Pr{ii,i2)Pr{ji,j2\ii,i2){xm{ii,i2) - £m(ji,j2))^- 

(20) 

Since the NQ is discrete, the integrals in (fT8])-(l20l) can be written as sums, the density function 
Py:{xi,X2) can be replaced by the point probabilities Pr{ii,i2) and p{z\ii{xi),i2{x2)) will be 
fully determined by the transition probabilities Pr (ji, ^2^1, ^2)- The distortion in (flTl) is a function 
of all three parameters A, c and a: ii depends on A, ^2 depends on A and c, and ji,j2 depends 
on A, c and a, as seen from (flOl) and (fTTI) . 

In the next section we propose a scheme where encoder 2 is continuous, i.e. a hybrid discrete 
analog scheme named Scalar Quantizer Linear Coder (SQLC). 
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B. Hybrid discrete-analog scheme: SQLC 

The motivation for introducing the SQLC in addition to the NQ are mainly given by the 
following two reasons: 1) The SQLC does not introduce quantization distortion at the encoder 
2, implying that the SQLC should improve over the NQ (as confirmed in Section ITVl) . 2) The 
SQLC is ideally simpler to implement since the encoder 2 basically consist of a limiter followed 
by scaling, whereas the NQ require two rounding operations. 

Encoder 1 is now a midrise quantizer with an even number of levels, where the representation 
values are transmitted directly on the channel. Encoder 2 is a limiter, denoted by £±k[-], that 
clips the amplitude of X2 to ±k, k E followed by scaling with a. That is. 



To simplify notation we denote centroid number i of encoder 1 by in the following. 

a and k must be chosen small enough in relation to A so we achieve the geometrical 
configuration shown in Fig. [3] (depicted for px = and 0.9), i.e. with non-intersecting channel 
segments. This will make it possible to uniquely decode both the quantized (depicted as dots) 
and continuous values (depicted as segments) from their sum. Note that when increases, the 
limitation to ±k becomes more and more insignificant since the joint pdf j5x(xi,X2) narrows 
along its minor axis, effectively "limiting" each source segment of the SQLC (the geometry of 
Py:{xi,X2) "limits" X2 given xi when px gets close to one), resulting in reduced distortion. 

Sequential decoding is again applied. First source 1 is recovered as 



where p{qi) takes into account that the midpoint of each channel segment shown in Fig. [3] 
changes with p^. That is, given that centroid qi was transmitted from encoder 1, the midpoint 
for the relevant channel segment becomes qi + E{ax2\qi} = qi + cxpxqi, and so 



We used the relation E{x2\xi} = pxXi |[T3l p. 233] for this calculation. Source 2 is then decoded 
as 




(21) 



gi{z) = argmin \\p{qi) - 



(22) 



p = qi{l + apx). 



(23) 



g2{z) = /3{z-gi{z)), 



(24) 
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Source space: A - 




Channel space: 



Source 
segment 



A =0.9 




9^(xi) + a(^±^[x2]) 



-4 -3-2 -1 I 1 f 2 3 - 4 

q-2 q-i /^i ^2 



Channel 
segment 



Source 
segment 



Fig. 3. SQLC concept for px = Q and 0.9. The dots represent encoder 1 while the line segments represent encoder 2. The 
numbers show how the segments in the source and channel space are related. If a, k and A are chosen appropriately in relation 
to each other, the channel segments will not overlap, and both sources can be decoded uniquely at the receiver. 



where (3 is an amplification factor. We further mimimize the MSE by computing 

Xm = ^{Xm\gi{z),g2{z)], 171=1,2. (25) 

We again formulate the optimization problem with an average transmit power constraint: 

min D. (26) 

a,/3,A,K:Pi+P2<2P 

To calculate D we could formulate similar integrals as for the NQ in Section IIII-A[ But 
since encoder 2 is now continuous, one will have to solve multiple integrals numerically. A 
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less computationally consuming approach, which also gives additional insight into the SQLC's 
underlying principle, is to divide D into five contributions: two from the encoding process 
and three due to channel noise. From the encoding process, we get quantization distortion 
for source 1 and clipping distortion eT^ for source 2. The effect of channel noise on source 1 
is only present when the centroids of encoder 1 are mis-detected, and we named it channel 
distortion ejj^^. Inspired by p^, pp. 62-98], we further divide the effect of channel noise on X2 
into additive noise, which name channel distortion £:^/j2' anomalous distortion e"^^, resulting 
from a threshold effect [[TSl . [fT6ll . that leads to large decoding errors. Here, threshold effect 
results when the wrong centroid for encoder 1 is detected since we will jump from one channel 
segment to another (see Fig. [3]). 

In order to calculate the effect of channel noise on the distortion, we first need the channel 
output pdf. We refer to Fig. |4] and [3] in order to explain the derivation of the pdf. 




Fig. 4. SQLC in source space wiien = 0.9. Tlie enlarged portion shows hiow to approximately calculate anomalous errors 
for X2 when px is high (>~ 0.7). li denotes the length of the portion of the X2 axis that contains the significant probability 
mass given xi, and e2 denote the minor axis of the ellipse shown (source space). 

1) Channel output pdf: We need to derive the pdf for two cases: 

Case 1: Assume that is small enough so the limitation to ±k is significant. Further let 
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1/2 = f2{x2), and m(-) be the Heaviside function. Then the pdf of 1/2 is 

1 

Pyiiy^) = ^= e 2^ti(y2 + aK)n(-y2 + +Po(5(2/2 - aft;) + 5(2/2 + a/t)), (27) 

v27racrT. 



where 



/■oo 

= Pr{x2 > K,} = Px{x2)dx2. (28) 



The product u(y2 + aK)u{—y2 + a/t) takes the clipping to ift; into account. Since samples 
with amplitude values outside [— k, k] are represented by ift; after clipping, one will get an 
accumulation of probability mass at ±k, hence the term Po{5{y2 ± ok)). Finally, a relation 
from [[T3I pp.131] was applied to determine the pdf of a random variable scaled by a. 

When /i and /2 are summed, the resulting pdf in (|27]) . become centered at the transmitted 
centroid Qi from encoder 1 (see Fig. [3]), and so the mean of the pdf, given g^, is equal to /i 
in (l23l) . Let 22 denote the received signal given g/. Assuming that the noise is additive (no 
threshold effects occurring for source 2), the resulting distribution after addition of noise is 
given by convolution lfT3l 181-182] 



PzAZ2)k =Py2*Pn 

/ Pi T T 



t-ai^ aVg(z2-M-y)j'+<T^y^ (29) 
/ e S^^^M +Po(j9„(z2 - /i +Pn(^2 - + ««;))• 

^ —OK 

Case 2: The pdf in (|29l ) must be modified when gets close to one, since clipping becomes 
negligible. We can now assume y2 = f2{x2) ~ 0!X2. When >> 0, each source segment will no 
longer be equivalent but contain different (but intersecting) ranges of y2 (seen from the = 0.9 
case in Fig. [3]). This implies that the channel segments shown in Fig. [3] are no longer exact 
copies of each other but describe somewhat different ranges of 1/2. Given that qi was transmitted, 
one can show that 

1 



p{y2\qi) = e ^^^^^W^) ^ (30) 

axa^2'K[l - pi) 

by using the expression for p{x2\xi) |[T3l p.223], the scaling of a random variable [fT3l pp. 
131] and inserting ji from (|23l) . After addition of noise, the pdf is given by the convolution 

^221(^2)7 = p{y2\(li) *Pn{n) (assuming that threshold effects are absent) 

e ^ (o:^CT^(p^-i)+CTi;) ^'x^\ 
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The validity of (|29l) and (|3T| ) must be determined. Fig. |4] provides a geometrical picture for 
the following discussion. Let li denote the length of the portion of the X2 axis that contains the 
significant probability mas^ given xi (or gj). li = 2v^||e2|| = 2by/WX2, where ||e2|| = b\^X2 
denote the length of the minor axis of the ellipse depicted in Fig. |4](the source space), b (~ 4) 
is a parameter determining the width of the ellipse shown, and should be chosen so that the 
significant probability mass is within this ellipse. 1!) = (/i/(2||e2||))^ and depends on p^. When 
Px = then 13=1 since the source space is rotationally invariant (a circle). When >~ 0.7 
then 13 ^ 1/ cos^Oj) = l/cos2(7r/4) = 2. That is, 'd e [1,2]. The pdf in (HH) is therefore valid 

when li > 2k while (|3TI) is valid when li < 2k. The total channel output pdf is then given by 

00 

p,{z) = Pr{q^)pUz) (32) 

i=— 00 

where Pz2i{z) is either (|29l) or (|3T1) depending on whether li > 2k or not. To calculate the 
distortion, one only need to consider (|29|) or (|3T1) centered at the origin, as explained later. 

2) Distortion and power calculation for source 2: The distortion for source 2 consists of 
three contributions: clipping distortion, channel distortion and anomalous distortion. 

Clipping distortion: Distortion from clipping, ef^, results whenever \x2\ > k. That is, an event 
with probability Pr{ |x2| > k} and resulting error (x2 — k^. Therefore 

POO 

el = 2 {x2- Kfpx{x2)dx2. (33) 

J K 

Channel distortion: Consider the effect of channel noise in the absence of threshold effects 
and let X2 = i±{x2) denote the clipped source. Then 

4h,2 = E{{X2 - {ax2 + n)m ^ ^'(1 - + ^'^l (34) 

where the last approximation comes from assuming that E{xl} = a^.. 

Anomalous distortion: When the wrong centroid from encoder 1 is detected, large decoding 
errors result for source 2. In the worst case, large positive and negative values are interchanged. 
This can be seen from Fig. [3l The magnitude of the error depends on whether li > 2k or not 
and is most severe when p^ = 0. 

^By "significant probability mass" we mean that all events except those with very low probability are included. 
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Consider first tliat li > 2k: The probability for the event Pr{x2 + n > A/2}, which result in 
anomalies for X2, is equal for each channel segment, given q^, and can therefore be calculated 
by assuming that Qi = 

Pthi = Pr\y2 + n> ^(1 + ap^) \ = 2 [ Pz2iz2)K\qi=odz2, (35) 

L ^ J ^f(l+apx) 

where Pz2{z2)k is given in (|29l) . The error is bounded by (2k)^, since k is detected as —k when 
neighboring segments in the channel space first start to intersect. 

Now consider the case when li < 2k. (|3TI) is just shifted according to which qi was transmitted 
from encoder 1, implying that the probability for anomalies is the same for each channel segment 
given qi. I.e. the relevant probability can again be calculated by assuming that qi = 0: 



Pth2 = 2 / Pzi{Z2)^\q,=odZ2, (36) 

(1+ap,) 

where Pzi{z2)y is given in (|3T1) . We now need to determine the magnitude of the anomalous 
errors 7. Since 7 is approximately the same in magnitude no matter which channel segment we 
"jump from" (see Fig. |4l), we choose to calculate 7 by considering jumps between the segments 
closest to the origin in the source (and channel) space. Then the parallelogram shown to the 
right in Fig. |4] can be used to approximately determine 7. Since ^ = 7r/4, the parallelogram 
consists of a square and two right triangles with both edges equal to A, implying that 



7 



/i - A = 26a, V^?(l-p.) - A. (37) 



where -d ^ 2 since px is close to one (see Section IIII-Blb . 

Since pthi and pth2 are the same for each segment, the anomalous distortion becomes 

{4ptw'^^ ^1 > 2k, 
(38) 
P^/^27^ h < 2k. 
Power: The transmitted power from encoder 2 is 

/oo paK 
y2Py2iy2)dy2 = ylpy^{y2)dy2 + 2poa'^ K^ , (39) 
00 J —an 

where Po is given in (l28l) and the term 2poa'^K^ accounts for the accumulation of probability 
mass at ±k due to clipping. When k is large {p^ close to 1), then P2 ~ a^c^^- 
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3 ) Distortion and power calculation for source 1 : The distortion for source 1 consists of two 
contributions: Quantization distortion and channel distortion. 

Quantization distortion: We assume a large enough number of quantization levels so that 
the quantization distortion consists of granular noise only, and we get 

= 2 V / (xi- qiYp^{xi)dxi = 2 / ( xi - (i - 1)A - — ) p^(xi)dxi. 

J(i-1)A J{i-1)A \ ^ / 

(40) 

Channel distortion: Distortion from channel noise, e^^, only occur when X2+n > A/2. Since 
we are only interested in determining the optimal design parameters for the SQLC, we simplify 
the analysis by considering jumps to the nearest neighboring centroids only. The probability for 
this event is the same as for the anomalous distortion calculated in (l35l) and (l36l) . The error we 
get when two neighboring centroids are interchanged is A^, thus 

Pthi, h > 2k, 

4hi = A' <! (41) 

Pth2, h < 2k. 

Power: The average transmit power for xi is 

oo oo y A\^ Z"*^ 

Pi = 2 Vp, ■ g2 = 2 VpJ - 1)A + - , p^= P.(xi)dxi. (42) 

i=i i=i \ -^(^-1)^ 

4) Optimization: Most of the terms derived in section IIII-B2I and IIII-B3I can be given closed 
form expressions by applying the error function, except for terms including the integral in ( |35] ) 
which must be solved numerically. Numerical optimization is therefore necessary in order to 
determine the optimal parameters. Instead of solving the constrained problem in (l43l) . we choose 



to scale Xi by ^ = y 2P — P2{a, K)/ax prior to quantization so we get an unconstrained problem 
instead. Note that with this scaling we must modify the factor fi with + ap^/C) i^i (l23l) to 
decode correctly. The average distortion now becomes D = (Di + D2)/2 where 



Di{a,A,K) 



g'(A)+4MKA,K) 
2P-P2(a,K) 

-2/,.N I ^2 /„ n\ I ,2 



D2{a, 13, A, k) = eiin) + e'ch2ia, /3) + A, k). (43) 

The optimized parameters as a function of the channel SNR is depicted in Fig. [5] for = and 
Px = 0.95. Notice that when = 0.95, we get a smaller A and a larger a for a given channel 
SNR compared to the p^ = case, implying improved fidelity (SDR) when increases, k is 
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Fig. 5. Optimal parameters for the SQLC system when px ~ and 0.95 (P = 1 and ~ 1). The results shown assume a 
scaling of xi by \/2P — Pij^x prior to quantization. |5(a)| a, A and re. |5(b)| /3. 
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not plotted in Fig. |5(a)| for = 0.95, since it becomes irrelevant. Notice also that /3 fits quite 
well to the function l/{a + aa^) when SNR > 10 dB. It is therefore simple to find decoder 2 



once encoder 2 is known. The curves in Fig. |5(a)| can be given mathematical expressions using 
nonlinear curve fitting to e.g. exponential functions. With channel state information available 
one can adapt the encoders to varying channel conditions simply by using these functions. 



C. Comparison and extension of NQ and SQLC 

From Section IIII-AI and IIII-BI one can observe that the NQ and the SQLC have similarities 
as well as differences. The main difference lies in how the encoding is performed for X2'- The 
NQ is discrete and use a sawtooth-like mapping ^2(3^2) to limit the output of encoder 2 to a 
fixed bounded interval, whereas the SQLC is continuous and either clips the amplitudes of X2 
(at low px) or simply rely on the fact that Px{xi,X2) limits the amplitudes of X2 for a given 
when Pj. is close to one. However, if the different parameters of the two schemes (c. A, a and 
k) are chosen optimally (so that anomalous errors are avoided), the geometrical configuration 
of the NQ after correct reconstruction at the decoder is quite similar to a quantized version of 
the SQLC, at least at high SNR. One may therefore expect the optimal behavior of the NQ and 
SQLC to be quite similar, and that the NQ have a certain loss compared to the SQLC due to an 
additional quantization distortion term. 

Although the SQLC has better performance than NQ, the NQ will be advantageous when a 
digital system must be constructed. Further, by letting A —t- for the NQ in encoder 2 we get 
a hybrid discrete-analog scheme where 22 is replaced by a continuous sawtooth-like function, as 
that in [[TtI . The resulting scheme should at least improve over the NQ. The question is if this 
modified NQ approach can improve upon the SQLC. Further research is needed. 

In the boundary case px 1, we can let A —t- and c — 1 for the NQ and A — )■ 0, 
K — 7- 00 and a = 1 for the SQLC. This means that both schemes are reduced to a distributed 
linear mapping, i.e. uncoded transmission, which is the optimal communication strategy when 
px = lB Corollary IV.sj]. 

The SQLC and NQ can be optimized using the same procedure provided in this paper for any 
unimodal source distribution. The performance will depend on the tails of the distribution. That is, 

''For the NQ to reach the bound we must let Pi = 2P since 12 = when px — 1. This is achieved by setting a — \/2P. 
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heavier tails than the Gaussian should result in worse performance, whereas smaller tails should 
result in improved performance. Consider a uniform v.s. a Laplacian distribution for the SQLC: 
In the uniform case the SDR will increase for a given SNR compared to the Gaussian since 
the pdf is narrower. One can also avoid distortion from clipping since the uniform distribution 
has compact support. In the Laplacian case, one must expect a loss in SDR compared to the 
Gaussian case, since large amplitude values have higher probability. This will result in a larger 
clipping distortion and/or a lower resolution for the quantizer of encoder 1 . 

Both schemes can rather easily be extended to he multivariate case, i.e. when M sources 
are assumed. Also, increasing the codelength beyond zero delay is rather straight forward by 
applying vector quantizers and linear coders of dimension A^. 

D. High SNR analysis 

The upper bound dH) can not be achieved with either the NQ or the SQLC even when SNR 
—7- oo (except when p = 1). One reason is that both schemes apply scalar quantization which is 
sub-optimal. Both schemes do, however, exhibit a constant gap to the bound as the SNR—)- oo. 
We will quantify this gap for the SQLC in the following. The reason why the NQ displays 
similar behavior will be briefly explained afterwards. 

The analysis here is approximate since, contrary to the case of infinite code length, there is a 
significant variance around the mean length of any stochastic vector due to short code length [[TSl 
p. 324] (for a normalized i.i.d. Gaussian random vector x of dimension N we have Var{||x||} = 
2a^/N). Further, to find closed form expressions that we can analyze further, we do not take 
all distortion terms into account, only the ones dominant under close to optimal conditions at 
high SNR: We can (nearly) avoid anomalous errors by assuming a distance A > ^/a'^lf + 6^ 
between each centroid in Fig. [3] (the distance is actually A(l + ap^) > \/ oi^l\ + 5^,, but since 
A(l + oipn^ > A, our assumption is still valid. The extra term is also of little significance when 
the SNR is large since a is very small), li = 26^ — p^), as shown in Section IIII-B 1 [ and 
Sn = '^bnO'n, whcrc 6„ (^ 4) is a constant that must be chosen so that the significant probability 
mass of the noise n is within 26„cr„. Assuming that clipping gets negligible when SNR grows 
large and noting that (5„ — t- as SNR— t- oo, the distortion can be approximated by 



D 



A2 <ylbHa\l-p,) 




(44) 



12(2P-P2) 3(2P-aV2) ' 
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We have scaled Di by 2P — a to satisfy the average power constraint, and applied the high 
rate approximation for a scalar quantizer. For D2 we have used the high SNR approximation 
(ML decoding) of (|34l ). By solving d\Di + D2\^/da = with respect to a^, we get 



" a,{2bHSNR{l- p,)-3al) a,6^SNR(l - p,) ' ^ ' ^ ^ 

where SNR= P/a^, and the last approximation results from removing constant terms at high 
SNR. We assume that cr^, = 1 in the following for simplicity. By inserting (l45l) in (|44|) and 
assuming that the SNR is very large, one can show that 

1 _3(2P/«2-l) 66^?SNR(l-pJ 



Di 62^(1 -p^) 62^(1 -p^)^6i9SNR(l 



1 _ ^ SNRv/6i?SNR(l -p^) _ ^3 /2SNR 




px ^ 1, 

(46) 



Since the upper bound can be approximated by SDR[/b ^ a/2SNR/(1 — p^), px 1, when 
SNR is large the loss from the bound is approximately quantified by SDRjoss ~ \/3/{b\^). 
By inserting 6^4 and i!) ^ 1 when p^. is close to zero and d ^ 2 when p^. is close to one, we 
find that the distance to the upper bound is around 3.5 dB when p^. = and around 5 dB when 
Px = 0.95. This estimate is somewhat pessimistic when p^ is close to zero (by ^ 0.5 dB) but 
relatively accurate when p^ is close to one (will be evident from the simulations in Section HVT). 
The estimate anyway indicates that there is a constant gap to the bound when the channel SNR 
grows large. 

Since the SQLC has a constant gap to the bound and the NQ is similar to a quantized version 
of the SQLC under optimal conditions (see Section IIH-CI) it naturally follows that also the NQ 
exhibit a constant gap to the bound. The gap will be somewhat larger since the NQ has an 
additional quantization distortion term. A similar method as that derived in llT9l 83-102] may be 
applied to quantify this gap, but is rather involved and will not be included here. 

IV. Simulations 

We compare the NQ and SQLC to the collaborative S-K mappings, the performance (SDR) 
upper bound from Section IlLBl uncoded transmission and the SSCC bound (derived in |[6|). 

We consider both an average- and equal transmit power constraint, where P = 1 in both 
cases. We will further assume that ax = I and use the optimal parameters resulting from the 
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optimization problem in ( fTSl ) for the NQ and the optimal parameters from Section IIII-B4I for the 
SQLC. Note that for Uncoded transmission, S-K mappings and the SSCC bound, Pi = P2 = P 
leads to the best possible performance under an average transmit power constraint. 



A. Average transmit power constraint 

Simulation results are shown in Fig. [6] for Px = ^ and = 0.95. 



When = (Fig. |6(a)[ ) the SQLC is about 2-3 dB away from the performance (SDR) upper 
bound and the NQ is inferior to the SQLC by about 0-2 dB. Both schemes are significantly 
better than uncoded transmission, but inferior to the S-K mapping, which is only 1-1.5 dB away 
from the upper bound. One reason why the SQLC backs off from the S-K mapping is that the 
S-K mapping is continuous and therefore completely avoid threshold effects [11, pp. 30-32]. This 
leads to a better utilization of the channel space, since the SQLC must leave an empty interval 
between each channel segment in order to avoid threshold effects. When = 0.95 (Fig. |6(b)| ) 
the performance of all schemes improve in SDR. The gap to the upper bound in terms of SDR, 
however, becomes larger for the S-K mapping (^ 2 — 3dB), SQLC (~ 2.5 — 5dB) and NQ 
(^ 3 — 7dB). The contrary is true for uncoded transmission. Considering that the mappings are 
delay free, the performance is still quite good. Interestingly, both the SQLC and NQ improve 
with increasing p^ without modification of the basic encoding and decoding structure, i.e only 
the parameters A, k, a, (3, c need to be changed. 

Robustness plots are also displayed in Fig. [6l The black dots mark the designed SNR: 29 dB 
for NQ and 37 dB for SQLC when p^ = 0, and 37 dB for NQ and 41 dB for SQLC when 
Px = 0.95. Both schemes improve and degrade gracefully under a channel SNR mismatch. 



The gain from increasing correlation as a function of p^. is shown in Fig. |7(a)| for the SQLC and 
uncoded transmission at 30 dB channel SNR. Note that the gain for the SQLC is not significant 
before px >~ 0.7, whereas the gain gets large when p^ — )■ 1. Uncoded transmission shows an 
even greater gain, which is natural since it goes from being highly sub-optimal when p^ = 
to achieve the bound for all SNR when p^. = 1. The gap to the performance upper bound as a 
function of px is plotted for NQ, SQLC, uncoded transmission and SSCC bound in Fig. |7(b)[ 
for 30dB channel SNR. Note that the distance to the upper bound is largest for both the NQ 
and SQLC when px is around 0.8 and that SQLC, NQ and uncoded transmission all reach the 
upper bound in the limit px ^ I, whereas the SSCC bound does not. 
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Fig. 6. Comparison of relevant schemes with an average power constraint. The dots shows the design SNR for the robustness 
plots. |6(i)]p, = 0.[6(b)]p, = 0.95. 
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Fig. 7. How correlation affects performance. |7(a)| Gain from correlation for the SQLC and uncoded transmission. |7(b)| Loss 
from OPTA as a function of px for 30 dB channel SNR. 
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B. Equal transmit power constraint 

Under an equal transmit power constraint, there is a loss in performance for both SQLC and 
NQ since they have asymmetric encoders. To simulate equal transmit power we use time sharing: 
Each source is encoded by /i(-) half the time and /2(-) the other half. Averaged over a large 
number of samples, then Pi = P2 = P h achieved. Fig. [8] shows the loss in performance for the 

Loss with equal transmit power constraint 
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Px 

Fig. 8. How an equal transmit power constraint Pi = P2 affects performance for the SQLC. The loss is compared to the 
average power constraint case with optimal Pi and P2. The loss is shown as a function of px when the channel SNR= 30 dB. 

SQLC with an equal power constraint compared to an average power constraint as a function of 
Px at 30 dB SNR (a similar effect is observed for the NQ). Notice that the loss becomes less 
as Px increases since the difference in power Pi — P2 becomes smaller. When = I, no loss 
is observed since the two encoders are equal. 

V. Summary and future Research 

In this paper, distributed delay free joint source-to-channel mappings for a bivariate Gaussian 
communicated on a Gaussian multiple access channel (GMAC) were proposed. We optimized 
a discrete mapping based on nested quantization (NQ) and a hybrid discrete-analog scheme 
SQLC which incorporates a piecewise continuous mapping. Both schemes are well performing 
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and improve on uncoded transmission for most SNR values outside the domain where uncoded 
transmission is optimal, thereby closing some of the gap to the performance upper bound. Since 
the NQ and SQLC have asymmetric encoders, a certain loss is observed when an equal transmit 
power constraint is imposed. Both NQ and the SQLC improve with increasing without 
changing the basic structure of the encoders and decoders, and achieves the performance upper 
bound when I. 

A collaborative scheme using Shannon-Kotel'nikov mappings was also discussed to provide 
an indication on where the bound for zero-delay coding may lie. Naturally, a back-off from this 
scheme was observed for both the SQLC and NQ. The main reason being that better mappings 
can be constructed when the encoders cooperate, i.e. mappings that make it possible to avoid 
so-called threshold effects. The SQLC is not necessarily the optimal distributed scheme either 
which may result in an additional loss factor. 

In our current research, we consider multiple sources as well as different source statistics and 
different attenuation for each sub-channel of the GMAC. One approach may be a generalization 
of the NQ and SQLC scheme. It would also be beneficial to determine if there exists better ways 
of doing zero delay distributed coding than the NQ and SQLC. 

Identifying the optimal performance of a communication system with a finite dimensionality 
constraint is crucial in assessing the performance of the proposed JSCC schemes. It is, however, 
a very difficult problem since many of the standard tools in information theory rely on infinitely 
long codewords. A generalization of work such as EOl could be an important step towards 
finding the performance bounds under such constraints. 



Using Xm and the auxiliary variables Xm and Xm from (fT6l) . Dm, m G {1, 2}, can be written 



= E{{Xm - Xmf} + E{{Xm - Xmf} + E{{Xm - X^f} + 2E{{Xm " Xm){Xm - Xm)} (47) 



Appendix 




+ 2E{{x, 
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For the first cross term we get 



since tlie last integral is (since the integral is over xi,X2 E ii,i2)- For the second cross term 
we get the same integral since 

{Xm - Xm{il,i2)){Xm{il,i2{i2)) - Xm{jl, j2))dXidX2 = 



PriJi, j2\ii,i2){xm{ii,i2{i2)) - Xm{ii,i2)) ■ ■ 

il,i2,jl,j2 

Pyi{xi,X2){Xm - Xm{il,i2))dXidX2 = 0. 

' xi,X2£ii,i2 

Finally, studying the third cross term we get 



(49) 



E{{X„, - Xm){x„, - X„,)} = ^ // pA^1,X2)--- 

n,i2,ii,i2-^-^^i'^2eii,i2 

Pr{jl,j2\il,i2){Xm{il,i2) - 5m(^l, «2(^2)))(5m(«l, «2(«2)) " ( jl , J2) )dXidX2 = 

^ Pr{ii,i2)Pr{jlj2\il,i2){Xm{il,i2) - 5m(^l, ^2(«2)))(5m(^l, ^2(^2)) - Xm{jl,j2)) = 
ii,i2,ji,j2 

'^Pr{ii,i2){Xm{il,i2) - 5m(^l, ^2(^2))) ^ -P?^(jl, J2K1, «2)(5m(«l, ^2(^2)) - Xm[h-,k)) = 0- 
ii,i2 31,32 

(50) 

due to the choice of Xm- What is left is then the three terms in (flTl) given by the three 
integrals (fTSi) . (fT9l ) and (|20|) . Hence, we conclude that Dm = Sq^m + ^c,m + ^n,m- 
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